Interactive audio-visual system

ABSTRACT

The invention concerns an interactive audio-visual system between a local scene and a remote scene, comprising at least a device ( 1 ) for producing images of the local scene and a device ( 2 ) for reproducing the image of the remote scene in an image plane ( 1 ). The image sensing device ( 1 ) comprises at least a camera ( 12 ) and a mirror ( 11 ) capable of reflecting on said camera ( 12 ) the parallel rays derived from the local scene and perpendicular to the image plane ( 1 ). Preferably, the mirror ( 11 ) is concave and has a focal point and the camera ( 12 ) is located in the proximity of the focal point of said mirror ( 11 ).

The invention concerns an interactive audiovisual system between a localscene and a remote scene, comprising at least one filming device for thelocal scene and a device for reproducing the image of the remote scenedestined to the local scene.

The field of the invention is that of telepresence, permitting a localscene and a remote scene to be presented.

When one or more persons are part of local and remote scenes, then theterm videoconference is used to allow these people to meet at adistance.

Telepresence is an extension of the videoconference.

The telepresence, like the videoconference, is obtained via theintermediary of the transmission of the image, sound and even datarepresenting texts, graphs, slides, etc.

Even though a scene may take place outdoors, it generally takes place ina building. This is why we will designate by the term local room theplace where the local scene will take place and remote room the placewhere the remote scene will take place.

The transmission of the image of a local room SL disposing of a filmingdevice 1 to a remote room SD disposing of a reproduction device followsa path schematically represented in FIG. 1, comprising a filming device1 such as a camera, possibly an analogue to digital converter CAN, acoding system C, a transmission network R, a decoding system D, possiblya digital to analogue converter CNA, and an image reproduction device 2such as a projector P linked to a plasma screen, a LCD, a CRT, etc.

In order to have mutual communication the chain shown in FIG. 1 isreproduced inversely in the remote room SD where the filming takes placethat is sent to the local room SL where the reproduction takes place.

Each room is finally equipped with both a filming device and areproduction device.

Later hereafter we will designate by the term interactive audiovisualsystem, a system located in a room that comprises at least a filmingdevice and a reproduction device.

Among the interactive audiovisual systems, the videoconferencing systemsare known. These videoconferencing systems are available in differentforms: videoconferencing rooms, personal computer videophonecommunicating multimedia, interactive terminals, etc.

These systems are intended to be used at a certain distance, both asconcerns the filming and the reproduction, depending on the size of theimage and the use that is made of it.

Close viewing is a predominant factor to ensure that those participatingin the videoconference or telemeeting viewing comfort anduser-friendliness, which guarantees the effect of telepresence. Thisclose viewing especially permits the impression of proximity between theremote participants to be enhanced by helping eye contact.

However the closer the scene to be filmed is close to the camera, thewider the camera angle needs to be. The widening of this angle whenfilming close-up poses a problem as illustrated in FIGS. 2 a and 2 b.

FIG. 2 a shows schematically an elevation view of a filming device 1,located in a local room SL, filming a local scene, represented by thelocal participants PL sat around a table positioned for example ataround 1 m from the camera disposing of a large field angle α. Thedirection in which the participants are looking is indicated by thesmall loop that is supposed to represent the participant's nose. On ascreen E, the image of the remote participants PD is formed, inparticular the image of the remote participant d.

In the local room SL, when a local participant a who is not in line withthe camera represented by the beam b1 talks to a remote participant, helooks at the image d′ of this remote participant on the screen E. Eventhough a looks directly at d′ along the beam ad′, the camera receivesthe beam a1 and finally films the participant from the side profile.

It is this side profile image that is transmitted to the reproductiondevice located in the remote room SD, which reproduces for d′, as showin FIG. 2 b, the image a′ of a as if a was not looking at d. Eye contactis not reproduced. This is called the parallax effect.

The term image plane is used to describe the plane in which the image ofd′ is situated. In this example, it is the same as the screen, but thisis not always the case as we will se later. The beam ad′ is a beam fromthe local scene to be filmed situated in a plane called the object planeand which is perpendicular to the image plane.

If, as shown in FIG. 2 c, the filming is carried out in cylindricalprojection mode retained in descriptive geometry, also called Mongegeometry or even in industrial drawing, allowing the camera to captureall of the beams parallel to ad′, and not in conical projection mode atan angle α as shown in FIG. 2 a, then the parallax effect would beeliminated.

The parallax effect has been presented in the context ofvideoconferencing but we can generalise by considering as a local scenepeople standing up or by no longer considering a person a but an object,for example a cube whose sides are blue and red and which is positionedat an angle: it has a red side and a blue side facing d′. However, thea1 beam can only come from a blue side, the image a′ of the cube willonly show the blue side instead of showing the blue and red sides of thecube positioned at an angle.

Furthermore, the size of the participants or the objects during thereproduction varies according to the part of the field in which they aresituated, and according to the field angle α of the camera. If severalpersons or objects are to be filmed, the field needs to be widened butthe images reproduced are somewhat curved in that the perspectiveeffects are distorted at the edges of the image; the images reproducedhave a variable magnifying effect as illustrated by FIGS. 3 a and 3 b,which is well known to photographers using wide angle lenses.

In each of these figures, the two local participants PL, a and b, arerepresented, one further away than the other, and filmed by a filmingdevice with, in the case of FIG. 3 a, a field angle α1 that is narrowerthan the case α2 of FIG. 3 b. The images reproduced a′1 and a′2 of a arevery much similar in both cases, but the magnification of b′1 inrelation to b (FIG. 3 a) is greater than that of b′2 in relation to b(FIG. 3 b).

A solution permitting the unwanted effect to be reduced, caused byfilming with a wide-angle lens (parallax effect and distortedperspective), consists of limiting the angle β formed in one point a ofthe scene by the beams ad′ and a1, represented in FIG. 2 a. Aspecification of the ETSI, European Telecommunications StandardsInstitute, recommends that this angle β does not exceed 8°. This angleis obtained by limiting the scene filmed or by cutting off part of it orby placing the camera close to the line of view, which is detrimental toviewing.

To compensate this disadvantage, several filming devices 1 are availableas shown in FIGS. 4 a and 4 b. In order to ensure the continuity of thefilming, these devices, whose optical axes are parallel and in a sameplane, are adjacent: several devices 1 are concatenated. However theresult is that the film fields have common or overlap zones ZR and thatthe images reproduced on one or more adjacent reproduction devices eachcorresponding to a filming device, will have discontinuities arisingfrom the duplicated or overlapped parts. The term viewing overlap themultiple reproduction of the same scene filmed by different cameraswhose fields partially overlap.

In the case of two filming devices 1, as shown in FIG. 4 a, there willbe a zone ZR with two overlaps; in the case of three filming devices 1,as shown in FIG. 4 b, there will be zones ZR with two overlaps whenclose to the filming devices, with three overlaps when further away,etc.

This overlap phenomenon of the views increases when the field angle ofthe filming devices increases.

Image processing software programmes are being developed at present toovercome this problem but they have not provided satisfactory results asyet.

The invention aims to overcome these problems linked to the parallaxeffect, the distorted perspective and possibly to the viewing overlap ofthe current videoconferencing systems by an interactive audiovisualsystem including a mirror that permits the filming device field angle tobe reduced, without limiting the scene.

A first approach consists of considering a filming device comprising amirror 11 and a camera 12 as shown in FIG. 5. The flat mirror 11 ispositioned opposite and at a distance D from the scene to be filmed sothat the camera 12 films the image of this scene in the mirror 11. Thescene to be filmed is the virtual image reflected by the mirror 11. Ifthe mirror 11 is flat, the scene to be filmed is twice the distance (at2D) from the camera 12, than if the camera 12 was positioned in theplace of the mirror 11. Such a mirror therefore permits the scene to bemoved further away artificially and therefore to reduce the field angleof the filming device, in this case the camera.

When the unit composed of the mirror 11 and the camera lens 12 istelecentric, the scene to be filmed is almost thrown back to infinityand the reproduced image of the entire (not reduced) scene filmed hasvirtually none of the faults previously described.

It should be remembered in fact that in a telecentric optical system STshown in FIG. 6, the principal beams issued from the object AB such asthe beam p are, at the output of the ST system, parallel to the opticalaxis zz′ of the said system.

Consequently, the size of the image A′B′ of an object AB is independentof the depth of observation; the point B″, the image of B along the beamp, is situated at the same distance from the O′z′ as B′. The image doesnot get bigger when an observer moves along the O′z′ axis. Of course theobserver will only have a clear image on a limited segment S of the O′z′axis situated around B′.

This also means that the size of the image of a lateral object will berespected in the same proportions as the size of an object close to theoptical axis of the ST system: for example, the size of the image B′C′of a lateral object BC will be respected in the same proportions as thesize of A′B′.

The ST system described is telecentric in the image space, in which thebeams are parallel; in the same way, a system may be telecentric in theobject space.

Hereafter, the notions of telecentrism and geometry (parallel,perpendicular, focus, etc.) are considered in the accepted limits forsystems with tolerances: they include the terms of near-telecentrism,near parallel, near perpendicular, near focus, etc.

The invention concerns an interactive audiovisual system between a localscene and a remote scene, comprising at least one filming device 1 forthe local scene and a device 2 for reproducing the image of the remotescene in an image plane I, principally characterised in that the saidfilming device 1 comprises at least a camera 12 and a mirror 11 capableof reflecting parallel beams from the local scene and perpendicular tothe image plane I to the said camera 12.

Preferably, the mirror 11 and the camera 12 form a telecentric unit.

According to one characteristic of the invention, the mirror 11 isconcave and has a focus and the camera 12 is situated close to the focusof the said mirror 11, which may be cylindrical, toric or revolution.

According to one characteristic, the reproduction device 2 comprises atleast a screen E onto which the image of the remote scene is formed anda semi-transparent mirror 21 capable of forming in the image plane I,the image of the screen E, the said semi-transparent mirror 21 beingplaced between the mirror 11 of the filming device and the local scene,so that the beams pass through the said semi-transparent mirror 21 andreach the mirror 11 of the filming device.

The semi-transparent mirror 21 may be flat or concave.

According to an additional characteristic, as the semi-transparentmirror 21 is concave, it has a focus and the system comprises amongothers a projector p situated close to the focus of the said mirror 21,which may be cylindrical, toric or revolution.

According to another characteristic, the reproduction device comprisesat least one projector p, projecting the image of the remote scene ontoa screen E, the mirror 11 of the filming device being capable of formingthe image of the screen E in the image plane I. it may comprise amongothers a flat mirror 21 situated and orientated so that it reflects theimage of the screen E onto the mirror 11 of the filming device.

The projector p may be situated at the focus point of the mirror 11 ofthe filming device. Similarly, the projector p, screen E andreproduction device mirror 11 unit may form a telecentric unit.

According to another characteristic, the system comprises severalfilming devices and/or several reproduction devices, that may be alignedor positioned along a convex curve.

The invention also concerns an interactive audiovisual system, between alocal scene and a remote scene, comprising at least one filming devicefor the local scene and a device for reproducing the image of the remotescene in an image plane I, characterised in that the said reproductiondevice 1 comprises at least one projector p and a screen E on which theimage of the remote scene is projected, and a mirror 21 capable offorming the image of the screen E in the image plane I.

Advantageously, the mirror is concave and the mirror 21, the projector pand the screen E form a telecentric unit.

The mirror 21 of the reproduction device may be of the cylindrical,toric or revolution type.

According to one characteristic, the mirror(s) are made of resin coveredwith a reflective surface.

Other specific features and advantages of the invention will becomeclear upon reading the description given by way of example and which isnot restrictive, and with reference to the appended drawings in which:

FIG. 1, already described, is a schematic representation of the chain ofelements situated along the journey of an image between a local room anda remote room,

FIGS. 2 a, 2 b and 2 c already described illustrate the parallax effect,

FIGS. 3 a and 3 b already described illustrate the magnifying effect ofthe reproduced image according to the field angle of the filming device,

FIGS. 4 a and 4 b, already described highlight the overlap zonesrespectively in the case of two and three filming devices,

FIG. 5, already described, schematically represents a filming devicecomprising a flat mirror,

FIG. 6, already described, schematically represents a telecentricoptical system,

FIG. 7, already described, schematically represents atelecentric-filming device according to the invention,

FIG. 8, already described, schematically represents a cylindrical mirrorwhich is used to obtain an asymmetrical telecentrism,

FIGS. 9 a, 9 b and 9 c schematically represent three embodiments of aninteractive audiovisual system according to the invention, the filmingand reproduction devices being uncoupled,

FIG. 10 schematically represents another embodiment of an interactiveaudiovisual system according to the invention, the filming andreproduction devices being coupled,

FIG. 11 schematically represents several filming devices aligned,

FIG. 12 schematically represents several interactive audiovisual systemsaccording to the invention, aligned.

The description is mainly situated in the context of a videoconferencewith participants benefiting from the telepresence effect, but is notrestricted to this specific case of telepresence.

In a first embodiment of the invention, represented in FIG. 7, weconsider a telecentric filming device 1 comprising a concave mirror 11placed opposite the local scene to be filmed, represented in this figureby the participants PL, and a camera 12 capable of filming the imageproduced by this mirror 11. The lens of the camera can be adjusted toimprove the telecentrism.

The mirror is made following a curve with a focus and the camera issituated at the focus point of the concave mirror; the mirror and thecamera thus form a telecentric unit. The focus may be situated in theplane of the scene to be filmed.

For simplification, the reproduction device is not shown in FIG. 7, theimage plane I is shown in which the local participants PL view the imageof the remote scene.

It will be considered hereafter that the plane(s) of the local sceneis/are parallel to the image plane I.

Therefore the parallel beams from the local scene to be filmed that areperpendicular to the image plane I, are all reflected by the mirror 11and converge at its focus point close to which the camera is positioned12, or more precisely the aperture of the camera lens.

The mirror 11 reflects to the camera 12 the beam composed of all of theparallel beams from the scene to be filmed. The mirror 11 can bereplaced by a transparent lens and the camera positioned at the focalpoint of the lens; however the beam would no longer be reflected and thefilming device would be too bulky.

In the case of FIG. 7, the beam ad′ which corresponds to the look of atowards d, which in the case of FIG. 2 a was cut off from the camerafield, will be included by the camera 12, in compliance with theproperties of this telecentric filming device 1.

As shown in FIG. 8, the telecentrism obtained may be asymmetrical, byusing a mirror 11 composed of a portion of a cylinder along an axis yy′.The cylinder may be circular or made to follow another curve with afocus point. The image obtained may be anamorphosed for specificapplications.

In this figure, the scene to be filmed is represented in an object planeO parallel to the plane xx′, yy′. Even though it is represented by thisplane O, the scene to be filmed may have a depth along the zz′ axis andthus be composed of several planes O.

A toric mirror may also be used, with different curve radii along thexx′ and yy′ axes; the cylindrical type mirror described above is aspecific case of a toric mirror, whose curve radius is infinite alongthe yy′ axis.

A revolution (spherical, parabolic, or other) type mirror may also beused, to obtain a revolution telecentrism.

FIGS. 9 a and 9 b show an interactive audiovisual system according tothe invention, comprising a telecentric filming device and areproduction device, operating separately.

The filming device comprises a concave and angled mirror 11, which is tosay that the axis between the mirror 11 and its focus is not the same asperpendicular LD of the O plane of the scene to be filmed, and a camera12 situated close to the focus of the mirror 11. Close to the focus isdefined as a small zone comprising this focus and the area around it.The mirror 11 and the camera 12 form a telecentric unit: the camera 12films the local scene in telecentric mode, reflecting on the mirror 11.

In FIG. 9 a, the reproduction device 2 comprises a projector (not shown)and a screen E in which the image of the emote scene is formed: thescreen E is not opposite the participants PL. For example, it may besituated in a plane perpendicular to the plane O of the scene to befilmed. The device 2 also comprises a flat semi-transparent mirror 21which permits the image of the screen E to be reflected to the imageplane I, destined to the local participants PL. The image plane I wherethe image of the screen is formed by the flat semi-transparent mirror 21does not coincide with the plane of the screen E. The flat mirror 21 isorientated so that the I plane is parallel to the O plane. For example,it is orientated at 45°.

The flat semi-transparent mirror 21 also allows the beams from the sceneto be filmed perpendicular to the image plane I to pas to the mirror.

In FIG. 9 b, the reproduction device 2 comprises a projector P and ascreen E onto which the image of the remote scene is projected; thescreen E is not opposite the local participants PL. The device 2comprises a semi-transparent cylindrical mirror 21 that allows the imageof the screen E to be reflected onto the image plane I, destined to thelocal participants PL. The mirror is orientated so that the I planer isparallel to the O plane.

Due to its cylindrical form, the mirror 21 enlarges the screen image tomake it the desired virtual size. It enlarges it in the tangentialdirection; in the generator direction, it acts as a flat mirror. Thecylindrical mirror 21 may also be used as for optical correction of thecamera, by correcting in particular aperture anomalies like a Schmittblade on a telescope.

The semi-transparent mirror 21 also allows the beams from the scene tobe filmed perpendicular to the I image plane to pass.

For example, we can consider for the semi-transparent mirrors 21 ofFIGS. 9 a) and 9 b), a transmission coefficient of 30% and a reflectioncoefficient of 70% favouring the luminosity of the image observed by thelocal participants. The camera 12 is sufficiently sensitive to remainefficient with a beam attenuated by 70%.

According to these embodiments, as represented in FIGS. 9 a) and 9 b),the filming and reproduction are uncoupled. The filming is telecentric.

When as shown in FIG. 9 b, the mirror 21, screen and projector P unitreturn to the local participants the reproduced image of the remotescene in a parallel beam, this is then telecentric; the reproductiondevice is then also telecentric. When the beam is not parallel asrepresented in FIG. 9 a), then the reproduction is not telecentric.

When the screen is flat, it does not contribute to the telecentrism ofthe reproduction device. When the screen is concave, the telecentrism isthen obtained by the screen, mirror and projector unit;

-   -   furthermore, if at the origin of the reproduced image formed on        the screen E, the filming has been shot with a telecentric        device, then the scene filmed will not have the defects linked        to the parallax effect, distorted perspective, etc. nor the        reproduced image of this scene. However, if at the origin of the        image reproduced the filming has not been shot with a        telecentric device, then the filmed scene may have the said        defects which will also appear in the image reproduced on the        screen.

According to another embodiment of the invention, represented in FIG. 9c), the filming and reproduction devices are uncoupled, but only thereproduction device is telecentric.

The acquisition device 1, comparable to that of FIG. 2 a), is nottelecentric. the reproduction device is composed of a projector p, ascreen E and a semi-transparent cylindrical mirror 21, with theprojector p being situated close to the focus of the mirror 21: thereproduction device is telecentric.

FIG. 10 shows another embodiment of an interactive audiovisual system ofthe invention, where the filming and reproduction are coupled.

The filming device comprises a concave mirror 11 disposing of a focusand a camera 12 situated close to this focus where the beams from thescene to be filmed and perpendicular to the image plane I converge, andwhere they are reflected back by the mirror 11 as described for FIG. 7.The camera may be situated in the plane of the scene to be filmed. Thepath of these beams is shown by a line with two arrows.

The reproduction device comprises a projector (not shown), a screen Eonto which the image of the remote scene is projected and a flat mirror22 receiving the image of the screen E.

The screen E may be a cardboard screen, a cathode or a plasma screen,etc. The mirror 22 is angled so that it reflects the image of the screenE onto the concave mirror 11 destined to the local participants PL. Thepath of the image from the screen E to the local participant PL is shownby a line with an arrow.

The mirror 22 may be removed and replaced by the screen E.

In this embodiment, the mirror 11 is used both by the filming device andby the reproduction device. The filming device, composed of the mirror11 and the camera 12 situated at the mirror focus 11 is telecentric.

The reproduction device is composed of a mirror 11, the screen E and theprojector p. Depending on whether the projector p is situated at thefocus of the mirror 11, (or at the focus of the mirror 11 and screen Eunit), the reproduction unit is telecentric or not.

Several telecentric filming devices may be aligned next to one anotheras shown in FIG. 11, thus ensuring the continuity of the scene beingfilmed. The filming devices may be contiguous or not.

Each filming device films a portion of the local scene. The view overlapphenomenon as shown in FIGS. 4 a) and 4 b) is considerably reduced asthe beams captured by the cameras 12 come from beams that are parallelto one another, or virtually parallel according to the telecentrismtolerances allowed by the system, which was not the case for the filmingdevices of FIGS. 4 a) and 4 b). The overlap zones will be virtuallynon-existent. The images reproduction by the reproduction devices thatmay be aligned, a reproduction device corresponding to a filming devicefor example, will have virtually no more discontinuity between them. Itis not necessary to have as many reproduction devices as filmingdevices.

The images filmed by the cameras may be concatenated beforereproduction, preferably by an image processing software programme, soas to form a single image to be reproduced. This image may even by splitonto several screens next to one another.

The filming devices and/or the reproduction devices may also bepositioned in a convex curve layout, for example in a half-circle.

Several interactive audiovisual systems such as that represented in FIG.9 b may be aligned as represented in FIG. 12. Each telecentric filmingdevice (camera 12 and mirror 11) films a portion of the local scene andthe local participants can observe the remote scene continuouslyreproduced from one reproduction device (screen E and semi-transparentmirror 21) to another.

Such interactive audiovisual systems of the invention may be used inother contexts than that of the videoconference.

The number of systems may be increased in diverse configurations so asto form a kiosk or a telepresence wall in a hall, a street, etc. linkedvirtually on a permanent basis to another remote kiosk or telepresencewall. It is no longer necessary to reserve the service as is often thecase at present for videoconference systems.

A person passing in front of a telepresence wall located in Paris forexample can communicate privately or informally with another person infront of a remote telepresence wall located in Nantes for example,connected to the one in Paris, as if they had met in the street, ion acorridor, etc. These remote people can walk “side by side” for example.

The mirrors may be made of resin covered with a reflective surface,possibly made of plastic.

1. An interactive audiovisual system between a local scene and a remotescene, comprising filming means for filming the local scene and areproduction device for reproducing the image of the remote scene in animage plane, wherein said filming means comprises several filmingdevices aligned side by side, wherein each filming device comprises atleast a camera and a mirror capable of reflecting parallel beams fromthe local scene and perpendicular to the image plane to said camera by asingle reflection, and wherein said mirror and said camera form atelecentric unit.
 2. The interactive audiovisual system according toclaim 1, wherein said mirror is concave and has a focus, and whereinsaid camera is positioned close to said mirror.
 3. The interactiveaudiovisual system according to claim 2, wherein the mirror of eachfilming device is a cylindrical, toric or revolution mirror.
 4. Theinteractive audiovisual system according to claim 1, wherein thereproduction device comprises at least a screen onto which the image ofthe remote scene is formed and a semi-transparent mirror capable offorming in the image plane, the image of the screen, saidsemi-transparent mirror being placed between the mirror of the filmingdevice and the local scene, so that the beams pass through saidsemi-transparent mirror and reach the mirror of the filming device. 5.The interactive audiovisual system according to claim 4, wherein thesemi-transparent mirror is flat or concave.
 6. The interactiveaudiovisual system according to claim 5, wherein the semi-transparentmirror is concave, wherein said mirror has a focus, and wherein saidsystem comprises among others a projector situated close to the focus ofsaid mirror.
 7. The interactive audiovisual system according to claim 6,wherein the mirror is cylindrical, toric or revolution.
 8. Theinteractive audiovisual system according to claim 1, wherein thereproduction device comprises at least one projector, projecting theimage of the remote scene onto a screen, the mirror of the filmingdevice being capable of forming the image of the screen in the imageplane.
 9. The interactive audiovisual system according to claim 8,comprising a flat mirror situated and orientated so as to reflect theimage of the screen onto the mirror of the filming device.
 10. Theinteractive audiovisual system according to claim 8, wherein theprojector is situated at the focus point of the mirror of the filmingdevice.
 11. The interactive audiovisual system according to claim 8,wherein the projector, the screen and the reproduction device mirrorunit form a telecentric unit.
 12. The interactive audiovisual systemaccording to claim 1, comprising several reproduction devices.
 13. Theinteractive audiovisual system according to claim 12, wherein saidreproduction devices are mutually aligned and wherein each reproductiondevice comprises at least one projector and a screen onto which theimage of the remote scene is projected, and a mirror capable of formingthe image of the screen in the image plane.
 14. The interactiveaudiovisual system according to claim 13, wherein said mirror is concaveand wherein the mirror, the projector and the screen form a telecentricunit.
 15. The interactive audiovisual system according to claim 13,wherein the mirror of the reproduction device is of the cylindrical,toric or revolution type.
 16. The interactive audiovisual systemaccording to claim 13, wherein the mirror(s) are made of resin coveredwith a reflective surface.